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(57) Abstract 

The invention relates to a 
system for obtaining assistance in 
operating an apparatus. A televi- 
sion receiver is provided with a 
microphone and a dedicated oper- 
ator call button and connected to a 
communication network. When a 
user presses the button, a connec- 
tion is established to a help desk 
operator, enabling the user and the 
operator to speak to each other. 
The operator can make the televi- 
sion receiver show a picture of the ~" 
control panel of the apparatus for 
which assistance is requested, and 
generate control signals for visu- 
alizing operations on the control 
panel. An animated image of the 
operator is shown on the televi- 
sion receiver, controlled by the sound level of the operator's voice, in order to elucidate the source of the voice speaking to the user. 
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System for obtaining assistance in operating an apparatus. 
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The present invention relates to a help system for obtaining assistance in 
operating an apparatus, comprising a display system, a communication network and an 
operator terminal, the display system being provided with operator call means for 
establishing a connection to the operator terminal via the communication network in response 
to a user command. 

Such a system is known from the English abstract of JP 62-127967. Upon 
pressing an operator call button of the known system, an audio and video connection is 
established with a help-desk operator. The video connection enables the operator to support 
his explanation by gestures and pictures. 

A disadvantage of the known system, however, is that expensive hardware 
is required and in most cases the bandwidth needed for transporting the video information is 
not available. 

An objective of the present invention is to provide a help system which 
offers some of the benefits of a video connection without the need for expensive hardware 
and a broadband network. To that end, the help system according to the invention is 
characterized in that the display system is conceived to store and display a picture of control 
means of the apparatus, and to visualize an operation upon the control panel in response to 
control signals obtained from the communication network, the operator terminal being 
conceived to generate said control signals and send the control signals via the communication 
network to the display system. The bandwidth required for transporting the control signals is - 
much smaller than the bandwidth which would have been required for sending a video 
recording of the control panel which is operated by a help-desk operator. 

The display system may comprise a microphone for enabling a user of the 
apparatus to verbally communicate a problem to the operator terminal. Similarly, the 
operator terminal may be equipped with a further microphone, enabling the operator to 
verbally communicate with the user. Alternatively, the operator terminal could be equipped 
with a speech recognition system and an automatic answering system, enabling the operator 
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terminal to respond automatically to spoken messages from the user. Microphones may be 
dispensed with at all when the dialog is fully automated, e.g. involving screen menus or 
numeric key selections. 

Features of the visualization may be the usage of a pointer for indicating a 
5 particular control, and highlighting or animation techniques which are well known from 
graphical computer applications. 

An embodiment of the help system is characterized in that the display 
system comprises permanent storage means for storing the picture, and that the display 
system is conceived to show the stored picture upon receiving an identification signal from 

10 the communication network, the operator terminal being conceived to generate the 

identification signal and send the identification signal via the communication network to the 
display system. By storing the picture of the control panel permanently in the display system, 
no video information need be sent via the communication network, only control signals. 

A further embodiment of the help system is characterized in that the 

15 operator terminal is conceived to send the picture via the communication network to the 
display system, the display system being conceived to receive the picture from the 
communication network and store the picture in volatile storing means. This solution is more 
flexible, because also pictures of control panels can be shown which are not known in 
advance. This is particularly useful if the assistance concerns the operation of an apparatus 

20 other than the display system itself, e.g. a VCR or a microwave oven. 

A further embodiment of the help system is characterized in that the 
display system comprises further storage means for storing an image of an operator, and that 
the display system is conceived to show the image of the operator which is stored in the 
further storage means. This feature gives the user an additional indication that a connection 

25 with a help-desk operator has been established. 

A further embodiment of the help system is characterized in that the 
further storage means comprise a permanent memory for storing the image of the operator, 
and that the display system is conceived to show the image upon receiving a further 
identification signal from the communication network, the operator terminal being conceived 

3Q to generate the further identification signal and send the further identification signal via the 
communication network to the display system. By storing the image of the operator 
permanently in the display system, no video information need to be sent via the 
communication network. Of course, resemblance between the image and the operator cannot 
be guaranteed in this way. To alleviate this drawback, the operator terminal could comprise 
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additional means for sending a selection code, enabling the operator to choose the pictorial 
representation, e.g. a female face. 

A further embodiment of the help system is characterized in that the 
operator terminal is conceived to send the image of the operator via the communication 
5 network to the display system, the display system being conceived to receive the image of 
the operator from the communication network and store it in the further storage means. In 
this way, resemblance between the picture and the operator can be guaranteed. 

A further embodiment of the help system is characterized in that the 
further storage means contains a plurality of pictures, and that the display system comprises 

10 animation means for animating the image of the operator by alternately showing one of at 
least two of said pictures. By animating the picture of the operator, the suggestion is 
strengthened that it belongs to the interlocutor. A preferred embodiment is characterized in 
that the animation means are conceived to be controlled by at least the sound level of a sound 
signal obtained from the operator terminal, and that the animation means are conceived to 

15 alternate at least two pictures of an operator which differ at least in the state of the mouth of 
the operator. Techniques for animating pictures which are stored locally are known from 
teleconferencing systems, see for instance U5> 5,347,306. 

A further embodiment of the help system is characterized in that the 
operator terminal comprises controlling means for controlling the display system by sending 

20 further control signals via the communication network to the display system, and that the 
display system is conceived to receive and interpret the further control signals and operate 
accordingly. This is advantageous when the assistance concerns the operation of the display 
system itself, particularly if the display system is combined with another complex apparatus, 
e.g. a television receiver. Operations demonstrated by the operator need not be imitated by 

25 the user, but are executed automatically, which quickens the communication. 

A further embodiment of the help system is characterized in that the 
display system is a television receiver. As a result, it is not necessary to purchase a dedicated 
help system, because TV-receivers are available in most homes. In a preferred embodiment, 
the TV receiver is characterized in that the television receiver is conceived to mute the TV 

30 sound automatically during a communication between a user and an operator. This enhances 
the intelligibility of the operator's voice. 

The invention further relates to a display system for use in a help system 
according to the invention, and to a method of obtaining assistance in operating an apparatus, 
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comprising a step of establishing a connection to an operator terminal. 

These and other aspects of the invention will be apparent from and 
elucidated with reference to the embodiments described hereinafter. 

In the drawings: 

Figure 1 shows a diagram of a help system according to the invention, 
Figure 2 shows a diagram of a display system according to the invention, 

and 

Figure 3 shows pictures of an operator's face to be used for animation. 



Figure 1 shows an embodiment of a help system according to the 
invention. It comprises a display system 1 which is connected to a communication network 
10 and an operator terminal 2. The display system 1 comprises a television receiver 3, which 
is provided with a microphone 4 and an antenna 5. The television receiver 3 can be operated 
by means of the remote control 8, which comprises a help button 9 as an embodiment of the 
operator call means. Another embodiment could be a menu item in an on-screen menu, or 
any other mechanism for entering user commands. 

j 

On the screen of the television receiver 3 two images 6 and 7 can be 
presented, image 6 being an animation of an operator's face and image 7 being a picture of a 
control panel, e.g. the remote control 8. 

The operator terminal 2 comprises a computer 12 which is provided with 
a microphone 11 and connected to the communication network 10. A keyboard 13 is 
connected to the computer 12. On the screen of the computer 12 an image 14 of a control 
panel can be presented. 

Figure 2 shows a diagram of the television receiver 3. The signal obtained 
from the antenna 5 is processed by the tuner 15 and split into an audio signal and a video 
signal. The audio signal is further processed by an audio processor 16 and a loudspeaker 17. 
The video signal is further processed by a video processor 18 and presented on a screen 19. 
Communication means 20 are conceived to send audio signals to the communication network 
10 and to the audio processor 16. The communication means 20 can receive audio signals 
from the communication network 10 and from the microphone 4, and image and control 
signals from the communication network 10. Animation means 21 can load pictures from 
first storage means 22 and second storage means 23 and send them to the video processor 18. 
When a user presses the help button 9, the television receiver 3 
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establishes a connection to a help-desk operator via the communication network 10. If no 
connection can be established a message is presented on the screen 19. The operator is 
equipped with the computer 12. From the start of the communication between the user and 
the operator, they can talk to each other, using the microphones 4 and 11. At the start of the 
5 communication, three pictures are sent from the operator terminal 2 via the communication 
network 10 to the television receiver 3, which stores them in the second storage means 23. 
The pictures represent three images of the operator's face, as shown in Figure 3. Picture 24 
shows the operator with the mouth closed, picture 25 shows the operator with the mouth half 
open, and picture 26 shows the operator with the mouth open. After receiving the three 
10 pictures, the animation means 21 cause the pictures to be presented in an alternating way as 
the image 6 on the screen 19. The alternation is controlled by the volume of the sound 
produced by the operator's voice. If the volume is high, picture 26 is shown, if the volume is 
low, picture 25 is shown, while during silence picture 24 is shown. Instead of alternating 
complete pictures, the image may be composed of a static part, e,g. a background picture, 
15 and two or more dynamic parts which are alternately projected into the image. In an 
alternative embodiment the pictures are stored permanently in permanent storage means, 
replacing the second storage means 23. As a result, it is no longer necessary to send the 
pictures via the communication network 10. In this embodiment resemblance of the pictures 
to the operator's face cannot be guaranteed. The operator terminal 2 could be provided with 
20 additional means (not shown) for generating an identification signal for activating one of 
various pre-defined sets of pictures. For example, if the operator is a woman, she can 
activate a set of pictures which represent a woman's face. 

The help system according to the invention enables a picture 7 of a 
control panel to be shown on the screen 19. This can be the control panel of another 
25 apparatus, e.g. a VCR or a microwave oven, or a control panel of the television receiver 3 
itself, e.g. the remote control 8. After informing the operator as to which apparatus 
assistance is requested for, the operator can enter an identification code, identifying a 
particular control panel. A picture of that particular control panel is then sent from the 
operator terminal 2 via the communication network 10 to the television receiver 3, which 
30 stores the picture in the storage means 22. The animation means 21" can send the picture to 
the video processor 18, which causes it to be presented on the screen 19 as the image 7. The 
animation means 21 can perform animation techniques on the picture of the control panel, 
such as moving a cursor over the picture and visualizing the pressing of a button on the 
control panel, e.g. by highlighting techniques, which are well known from graphical 
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computer programs. This animation is controlled by control signals which are generated by 
the operator using the keyboard 14, or any other input means, while inspecting an image 14 
on the screen of the computer 12 which is substantially identical to the image 7 on the screen 
19 of the television receiver 3. 

In an alternative embodiment assistance can only be obtained for a pre- 
defined set of devices of which the control panels are stored permanently in the storage 
means 22. In that embodiment only the control signals for controlling the animation need be 
sent via the communication network 10 to the television receiver 3. In the most restricted 
case, only assistance concerning the television receiver 3 can be obtained, but in that case the 
operator terminal could be provided with control means (not shown) for controlling the 
operation of the television receiver 3 by means of control signals which are sent via the 
communication network 10. This enables the operator, besides explaining how a certain goal 
is achieved, to show the effect of the operations required for achieving the goal. Especially 
for complex tasks this could be a useful addition. 

Connections which would be established by accidentally pressing the 
operator call button can be avoided by asking the user for confirmation, e.g. by means of a 
screen message which asks the user to press the operator call button a second time. Other 
procedures, such as a time-out on the operator call button, or an interactive conversation 
using the screen, are well known and can be applied as well. 

If the television receiver 3 is producing sound just before the connection 
established, the sound is muted partially or completely, in order to enhance the intelligibility 
of the operator's voice. 

In summary, the invention relates to a system for obtaining assistance in 
operating an apparatus. A television receiver is provided with a microphone and a dedicated 
operator call button and connected to a communication network. When a user presses the 
button, a connection is established to a help desk operator, enabling the user and the operator, 
to speak to each other. The operator can make the television receiver show a picture of the 
control panel of the apparatus for which assistance is requested, and generate control signals 
for visualizing operations on the control panel. An animated image of the operator is shown 
on the television receiver, controlled by the sound level of the operator's voice, in order to 
elucidate the source of the voice speaking to the user. 
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1. A help system for obtaining assistance in operating an apparatus, 
comprising a display system, a communication network and an operator terminal, the display 
system being provided with operator call means for establishing a connection to the operator 
terminal via the communication network in response to a user command, characterized in 

5 that the display system is conceived to store and display a picture of control means of the 
apparatus, and to visualize an operation on the control panel, in response to control signals 
obtained from the communication network, the operator terminal being conceived to generate 
said control signals and send the control signals via the communication network to the 
display system. 

10 

2. A help system according to claim 1 , characterized in that the display 
system comprises a microphone, the display system being adapted to send signals obtained 
from said microphone via the communication network to the operator terminal. 

15 3. A help system according to claim 2, characterized in that the operator 

terminal comprises a further microphone, the operator terminal being adapted to send signals 
obtained from said further microphone via the communication network to the display system. 

4. A help system according to any one of claims 1 to 3, characterized in 

20 that the operator terminal comprises a speech recognition system for recognizing speech 
input from the display system and an automatic answering system for generating an answer 
in response to said speech input, the operator terminal being adapted to send said answer via - 
the communication network to the display system. 

25 5. A help system according to claim 1, characterized in that the control 

signals define the position of a cursor in the picture. 

6. A help system according to claim 1 or 2, characterized in that the 

control signals define temporary changes in at least parts of the picture. 
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7. A help system according to any of the claims 1 to 3, characterized in 
that the display system comprises permanent storage means for storing the picture, and that 
the display system is conceived to show the stored picture upon receiving an identification 
signal from the communication network, the operator terminal being conceived to generate 

5 the identification signal and send the identification signal via the communication network to 
the display system. 

8. A help system according to any of the claims 1 to 3, characterized in 
that the operator terminal is conceived to send the picture via the communication network to 

10 the display system, the display system being conceived to receive the picture from the 
communication network and store the picture in volatile storing means. 

9. A help system according to any of the claims 1 to 5, characterized in 
that the display system comprises further storage means for storing an image of an operator, 

15 and that the display system is conceived to show the image of the operator which is stored in 
the further storage means. 

10. A help system according to claim 6, characterized in that the further 
storage means comprise a permanent memory for storing the image of the operator, and that 

20 the display system is conceived to show the image upon receiving a further identification 
signal from the communication network, the operator terminal being conceived to generate 
the further identification signal and send the further identification signal via the 
communication network to the display system. 

25 11. A help system according to claim 6, characterized in that the operator 

terminal is conceived to send the image of the operator via the communication network to the 
display system, the display system being conceived to receive the image of the operator from- 
the communication network and store it in the further storage means . 

12. A help system according to any of the claims 6 to 8, characterized in 
30 that the further storage means contains a plurality of pictures, and that the display system 

comprises animation means for animating the image of the operator by alternately showing 
one of at least two of said pictures. 

13. A help system according to claim 9, characterized in that the animation 
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means are conceived to be controlled by at least the sound level of a sound signal obtained 
from the operator terminal, and that the animation means are conceived to alternate at least 
two pictures of an operator which differ at least in the state of the mouth of the operator. 

5 14. A help system according to any of the claims 1 to 10, characterized in 

that the operator terminal comprises controlling means for controlling the display system by 
sending further control signals via the communication network to the display system, and that 
the display system is conceived to receive and interpret the further control signals and 
operate accordingly. 

10 

15. A display system conceived to be used in a help system according to any 
of the claims 1 to 11, comprising operator call means and a microphone, and conceived to be 
connected to a communication network and to show a picture of at least a part of a control 
panel of an apparatus and to visualize an operation on the control panel, the visualization 

15 being controlled by control signals obtained from the communication network. 

16. A display system according to claim 12, characterized in that the display 
system is a television receiver. 

20 17. A television receiver according to claim 13, characterized in that the 

television receiver is conceived to mute the TV sound automatically during a communication 
between a user and an operator. 

18. A method of obtaining assistance in operating an apparatus, comprising 
25 the step of establishing a connection to an operator terminal, characterized in that the 

method further comprises the steps of: 

showing a picture of at least a part of a control panel of the apparatus, 
visualizing an operation on the control panel, the visualization being actuated by 

control signals which are sent by the operator terminal. 

30 

19. A method according to claim 15, characterized in that the method 
further comprises the step of showing an image of an operator. 



A method according to claim 16, characterized in that the method 
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further comprises the step of animating the image of the operator, controlled by at least the 
sound level of a sound signal obtained from the operator terminal. 



WO 99/30253 



PCT/IB98/01936 




FIG. 3 



PCT 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




(51) International Patent Classification 6 : 




(11) International Publication Number: 


WO 99/30253 


G06F 9/44, 17/60 


A3 










(43) International Publication Date: 


17 June 1999 (17.06.99) 



(21) International Application Number: PCT/IB98/01936 

(22) International Filing Date: 3 December 1998 (03.12.98) 



(30) Priority Data: 
97203892.1 



1 1 December 1 997 ( 1 1 . 1 2.97) EP 



(71) Applicant: KONINKLUKE PHILIPS ELECTRONICS N.V. 
[NIVNL]; Groenewoudseweg 1, NL-5621 BA Eindhoven 
(NL). 

(71) Applicant (for SE only): PHILIPS AB [SE/SE]; Kottbygatan 7, 

Kista, S-164 85 Stockholm (SE). 

(72) Inventors: VRIENS, Leonarda, J., M.; Prof. Holstlaan 6, 

NL-5656 AA Eindhoven (NL). KUIJPERS, Henricus, J., 
C; Prof. Holstlaan 6, NL-5656 AA Eindhoven (NL). 
STEEN, Marc, G., D.; Prof. Holstlaan 6, NL-5656 AA 
Eindhoven (NL). 

(74) Agent: FAESSEN, Louis, M., H.; Internationaal Octrooibureau 
B.V., P.O. Box 220, NL-5600 AE Eindhoven (NL). 



(81) Designated States: JP, KR, European patent (AT, BE, CH, CY, 
DE, DK, ES, FI, FR, GB. GR, IE, IT, LU, MC, NL, PT 
SE). 



Published 

With international search report. 
Before the expiration of the time limit for amending the claims 
and to be republished in the event of the receipt of amendments. 

(88) Date of publication of the international search report: 

26 August 1999(26.08.99) 



(54) Title: SYSTEM FOR OBTAINING ASSISTANCE IN OPERATING AN APPARATUS 
(57) Abstract 



£ 



10 



I 



18 



CTR 



-19 



The invention relates to a 
system for obtaining assistance in 
operating an apparatus. A televi- 
sion receiver is provided with a 
microphone and a dedicated oper- 
ator call button and connected to a 
communication network. When a 
user presses the button, a connec- 
tion is established to a help desk 
operator, enabling the user and the 
operator to speak to each other. 
The operator can make the televi- 
sion receiver show a picture of the — 
control panel of the apparatus for 
which assistance is requested, and 
generate control signals for visu- 
alizing operations on the control 
panel. An animated image of the 
operator is shown on the televi- 
sion receiver, controlled by the sound level of the operator's voice, in order to elucidate the source of the voice speaking to the user. 
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